Method for teeth segmentation and alignment detection in CBCT volume

ABSTRACT

A method of automatic tooth segmentation, the method executed at least in part on a computer system acquires volume image data for either or both upper and lower jaw regions of a patient and identifies image content for a specified jaw from the acquired volume image data. For the specified jaw, the method estimates average tooth height for teeth within the specified jaw, finds a jaw arch region, detects one or more separation curves between teeth in the jaw arch region, defines an individual tooth sub volume according to the estimated average tooth height and the detected separation curves, segments at least one tooth from within the defined sub-volume, and displays the at least one segmented tooth.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. Ser. No. 13/949,281 filed onJul. 24, 2013 entitled “METHOD FOR TEETH SEGMENTATION AND ALIGNMENTDETECTION IN CBCT VOLUME” to Chen, which published as US 2013/0308846;which itself did claim the benefit of U.S. Provisional PatentApplication Ser. No. 61/825,658 entitled “METHOD FOR TEETH SEGMENTATIONAND ALIGNMENT DETECTION IN CBCT VOLUME” filed on May 21, 2013 in thenames of Shoupu Chen et al. and itself was also a Continuation-in-Partof U.S. Ser. No. 13/448,466 filed on Apr. 17, 2012 entitled “METHOD FORTOOTH DISSECTION IN CBCT VOLUME” to Chen, which published as US2013/0022254; which is itself a Continuation-in-Part of U.S. Ser. No.13/187,596 filed on Jul. 21, 2011 entitled “METHOD AND SYSTEM FOR TOOTHSEGMENTATION IN DENTAL IMAGES” to Chen et al., the contents of which areall each incorporated fully herein by reference.

FIELD OF THE INVENTION

The present invention relates generally to image processing in x-raycomputed tomography and, in particular, to automatic tooth segmentation,teeth alignment detection, and manipulation in a digital CBCT volume.

BACKGROUND OF THE INVENTION

Schleyer et al. (“A Preliminary analysis of the dental informaticsliterature”, Adv. Dent Res 17:20-24), indicates a rise in the number ofdental informatics papers in journals such as Journal of the AmericanMedical Informatics Association, the Journal of the American DentalAssociation, and the Journal of Dental Education. Among topics surveyed,imaging, image processing, and computer-aided diagnosis were areas ofinterest.

Tooth image segmentation is of benefit for dental applications such ascomputer aided design, diagnosis, and surgery. Various approaches havebeen proposed in recent years to address tooth segmentation. However,researchers have noted the difficulty of tooth segmentation. Forexample, researchers Shah et al. describe a method for automatingidentification of deceased individuals based on dental characteristicsin comparing post-mortem images with tooth images in multiple digitizeddental records (“Automatic tooth segmentation using active contourwithout edges”, 2006, Biometrics Symposium). Other methods are describedby Krsek et al. in “Teeth and jaw 3D reconstruction in stomatology”(Proceedings of the International Conference on Medical InformationVisualisation—BioMedical Visualisation, pp 23-28, 2007); Akhoondali etal. in “Rapid Automatic Segmentation and Visualization of Teeth inCT-Scan Data”, Journal of Applied Sciences, pp 2031-2044, (2009); andGao et al. in “Tooth Region Separation for Dental CT Images”,Proceedings of the 2008 Third International Conference on Convergenceand Hybrid Information Technology, pp 897-901, (2008).

In orthodontia applications, apparatus, system, and methods have beendeveloped to facilitate teeth movement utilizing clear and removableteeth aligners as an alternative to braces. A mold of the patient's biteis initially taken and desired ending positions for the patient's teethare determined, based on a prescription provided by an orthodontist ordentist. Corrective paths between the initial positions of the teeth andtheir desired ending positions are then planned. Aligners formed to movethe teeth to the various positions along the corrective path are thenmanufactured.

US 2011/0137626 by Vadim et al. describes a method to construct an archform with the 3-dimensional (3-D) data of patient's teeth and facialaxis points for the teeth. However, the Vadim et al. '7626 method doesnot provide the ability to visualize maneuvering the teeth individuallyand digitally for treatment planning. With this and other methods, thedigital data obtained from the Cone-Beam Computed Tomography (CBCT)dental volume image is not associated with desired teeth movement or acorresponding treatment strategy. This limits the usefulness of thevolume image data.

Thus, there is a need for a system and method for automaticallysegmenting teeth from CBCT data, with tools for automatically analyzingtooth alignment and allowing a user to manipulate teeth digitally.

SUMMARY OF THE INVENTION

It is an object of the present invention to advance the art of toothsegmentation and analysis from cone beam CT (CBCT) images. A feature ofthe present invention is auto-segmentation of teeth without operatorintervention. According to an embodiment of the present invention, anautomated assessment of tooth alignment is provided, along with adisplay of alignment information.

These and other aspects, objects, features and advantages of the presentinvention will be more clearly understood and appreciated from a reviewof the following detailed description of the preferred embodiments andappended claims, and by reference to the accompanying drawings.

According to an aspect of the present invention, there is provided amethod of automatic tooth segmentation, the method executed at least inpart on a computer and comprising: acquiring volume image data foreither or both upper and lower jaw regions of a patient; identifyingimage content for a specified jaw from the acquired volume image dataand, for the specified jaw: (i) estimating average tooth height forteeth within the specified jaw; (ii) finding a jaw arch region; (iii)detecting one or more separation curves between teeth in the jaw archregion; (iv) defining an individual tooth sub volume according to theestimated average tooth height and the detected separation curves; (v)segmenting at least one tooth from within the defined sub-volume; anddisplaying the at least one segmented tooth.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features, and advantages of theinvention will be apparent from the following more particulardescription of the embodiments of the invention, as illustrated in theaccompanying drawings, in which:

FIG. 1 is a logic flow diagram showing processes for forming one or moreseparation curves according to an embodiment of the present invention;

FIG. 2 is a view of a set of reconstructed CBCT images having objects ofinterest;

FIG. 3A is a view of a set of reconstructed CBCT images having featuresof interest with concentric curves overlaid;

FIG. 3B is a schematic diagram that shows how a panoramic image isformed by unfolding a curved sub-volume;

FIG. 4 is a view of a set of reconstructed CBCT images having objects ofinterest with concentric curves and lines for unfolding formedperpendicular to the concentric curves overlaid;

FIG. 5A is a logic flow diagram that shows a sequence of steps forforming an averaged image W from a number of intensity images;

FIG. 5B is a logic flow diagram that shows a sequence of steps forgenerating and displaying a separation curve between a first and asecond object in a volume image;

FIG. 5C shows an imaging apparatus that is used for forming one or moreseparation curves;

FIG. 6A shows different views of a volume image;

FIG. 6B shows a volume image with an outlined tooth, obtained from userinterface input;

FIG. 6C is schematic diagram of a perspective view of vertical (coronal)slices;

FIG. 6D is an image that shows the use of profile lines for analyzingthe gap between teeth and generating a separation curve;

FIG. 7A is a plan view that shows how valley points are used to generatea separation curve;

FIG. 7B is a schematic diagram in perspective, illustrating findingvalleys on profile lines in the average image of an unfolded curvedslab;

FIG. 8 is an average image of an unfolded curved slab with theidentified separation curves overlaid;

FIG. 9 is a view of a reconstructed CBCT image having separation curvesmapped between the objects of interest;

FIG. 10 is a logic flow diagram that describes an auto-segmentationsequence according to an embodiment of the present invention;

FIG. 11A is a 2-D image that shows how average height is computed fromthe image content;

FIG. 11B shows a volume image to be processed;

FIG. 12 shows the image of FIG. 11B with the arch region identified;

FIG. 13 shows the image of FIG. 11B with separation curves identified;

FIG. 14 shows the image of FIG. 11B with individual teeth defined andoutlined;

FIG. 15 shows segmented teeth using an embodiment of the presentinvention;

FIG. 16 is a logic flow diagram that shows a sequence of steps forreporting alignment assessment;

FIG. 17 shows the segmented teeth of FIG. 15 with centroids identified;

FIG. 18 shows an arrangement of tooth centroids that conforms to aconvex hull;

FIG. 19 shows segmented teeth with centroids identified; and

FIG. 20 shows an arrangement of tooth centroids that do not conform to aconvex hull.

DETAILED DESCRIPTION OF THE INVENTION

In the following detailed description of embodiments of the presentinvention, reference is made to the drawings in which the same referencenumerals are assigned to identical elements in successive figures. Itshould be noted that these figures are provided to illustrate overallfunctions and relationships according to embodiments of the presentinvention and are not provided with intent to represent actual size orscale.

Where they are used, the terms “first”, “second”, “third”, and so on, donot necessarily denote any ordinal or priority relation, but may be usedfor more clearly distinguishing one element or time interval fromanother.

In the context of the present disclosure, the term “image” refers tomulti-dimensional image data that is composed of discrete imageelements. For 2D (two-dimensional) images, the discrete image elementsare picture elements, or pixels. For 3D (three-dimensional) images, thediscrete image elements are volume image elements, or voxels.

In the context of the present disclosure, the term “code value” refersto the value that is associated with each volume image data element orvoxel in the reconstructed 3D volume image. The code values for CTimages are often, but not always, expressed in Hounsfield units.

In the context of the present disclosure, the term “geometric primitive”relates to an open or closed geometric figure or shape such as atriangle, rectangle, polygon, circle, ellipse, free-form shape, line,traced curve, or other traced pattern.

The term “highlighting” for a displayed feature has its conventionalmeaning as is understood to those skilled in the information and imagedisplay arts. In general, highlighting uses some form of localizeddisplay enhancement to attract the attention of the viewer. Highlightinga portion of an image, such as an individual organ, bone, or structure,or a path from one chamber to the next, for example, can be achieved inany of a number of ways, including, but not limited to, annotating,displaying a nearby or overlaying symbol, outlining or tracing, displayin a different color or at a markedly different intensity or gray scalevalue than other image or information content, blinking or animation ofa portion of a display, or display at higher sharpness or contrast.

In the context of the present disclosure, the descriptive term “highdensity object” generally indicates a mass, or object such as a tooth,that exceeds the density of the surrounding materials (such as softtissues or air) and would be identified in a radiographic image as ahigh density object by a skilled practitioner. Because of differencesrelated to dosage, however, it is impractical to specify any type ofabsolute threshold for defining high density.

The term “set”, as used herein, refers to a non-empty set, as theconcept of a collection of elements or members of a set is widelyunderstood in elementary mathematics. The term “subset”, unlessotherwise explicitly stated, is used herein to refer to a non-emptyproper subset, that is, to a subset of the larger set, having one ormore members. For a set S, a subset may comprise the complete set S. A“proper subset” of set S, however, is strictly contained in set S andexcludes at least one member of set S.

In the context of the present disclosure, the term “dissection” relatesto methods used for separating one object from another, adjacent object.Thus, dissection of a subject tooth in an intraoral volume image definesa boundary between the subject tooth and an adjacent or neighboringtooth.

In the context of the present disclosure, the terms “viewer”,“operator”, and “user” are considered to be equivalent and refer to theviewing practitioner or other person who views and manipulates an image,such as a dental image, on a display monitor. An “operator instruction”or “viewer instruction” is obtained from explicit commands entered bythe viewer, such as using a computer mouse or touch screen or keyboardentry.

The subject matter of the present invention relates to digital imageprocessing and computer vision technologies, which is understood to meantechnologies that digitally process data from a digital image torecognize and thereby assign useful meaning to human-understandableobjects, attributes or conditions, and then to utilize the resultsobtained in further processing of the digital image.

Image segmentation is a process that partitions a digital image into adefined set of features so that image analysis can be simplified.Segmentation assigns a label to each image pixel so that pixels thathave the same label are considered to be a part of the same structure ortype of feature and can share common treatment. Due to factors such asthe relative complexity of the image content and difficulties presentedby the shape and structure of teeth, conventional attempts at toothsegmentation have not been sufficiently robust for widespreadapplication.

A number of known image segmentation methods familiar to those skilledin the image analysis arts, such as region growing, utilize “seeds”.Seeds are pixels that are either identified automatically from the imagecontent or explicitly identified by a user. As the name implies, theseed is used as a hint or starting point for a region growing process.In region growing, a region of the image is defined from its initialseed pixel(s) or voxels (s) by successively analyzing neighboring pixel(or voxel) values relative to the seed pixel(s) and incorporating apixel (or voxel) into a region when its difference from the seed pixel(or voxel) is below a predetermined value.

Akhoondali et al. proposed an automatic method for the segmentation andvisualization of teeth in multi-slice CT-scan data of the head in “RapidAutomatic Segmentation and Visualization of Teeth in CT-Scan Data”,Journal of Applied Sciences, pp 2031-2044, (2009). The algorithm thatwas employed consists of five main procedures. In the first part, themandible and maxilla are separated using maximum intensity projection inthe y direction and a step like region separation algorithm. In thesecond part, the dental region is separated using maximum intensityprojection in the z direction, thresholding and cropping. In the thirdpart, the teeth are rapidly segmented using a region growing algorithmbased on four thresholds used to distinguish between seed points, teethand non-tooth tissue. In the fourth part, the results are visualizedusing iso-surface extraction and surface and volume rendering. Asemi-automatic method is also proposed for rapid metal artifact removal.However, in practice, it is very difficult to select a total of fivedifferent threshold values for a proper segmentation operation. Theirpublished results show relatively poor dissection between the teeth insome cases.

Gao et al. disclosed a method to construct and visualize the individualtooth model from CT image sequences for dental diagnosis and treatment(see “Tooth Region Separation for Dental CT Images”, Proceedings of the2008 Third International Conference on Convergence and HybridInformation Technology, pp 897-901, 2008). Gao's method attempts toseparate teeth for CT images where the teeth touch each other in someslices. The method finds the individual region for each tooth andseparates two teeth if they touch. Their proposed method is based ondistinguishing features of the oral cavity structure. The use of full 3Ddata, instead of 2D projections, may cause loss of some information. Thedescribed method initially separates upper and lower tooth regions andthen fits the dental arch using fourth order polynomial curves, after aseries of morphological operations. The method assumes that there existsa plane separating two adjacent teeth in 3D space. In this plane, theintegral intensity value reaches a minimum. Along each arch point, thismethod obtains a plane and calculates the integral intensity. Thesevalues are then used to draw a profile. After analyzing all the localminima, this method obtains the separating point and the position of theseparating plane. The information for the tooth region can guide thesegmentation of both the individual tooth contours in 2D space and thetooth surfaces in 3D space. However, it appears that Gao's method maynot actually separate (or dissect) the teeth correctly; the separation(or dissection) curves that are obtained in many cases cut through theteeth region of interest in certain slices.

Referring to the logic flow diagram of FIG. 1, there is shown a sequenceof steps used for teeth dissection for a dental CBCT volume (accessed inan image access step 102) in one embodiment. A volume contains imagedata for one or more images (or equivalently, slices). An originalreconstructed CT volume is formed using standard reconstructionalgorithms using multiple 2D projections or sinograms obtained from a CTscanner. Normally, only a fraction or subset of the images that form thevolume contain high density objects and is selected for processing; therest of the CT reconstructed volume accurately represents soft tissue orair.

This selection of a subset of images for this procedure is done in animage selection step 104. A number of neighboring high density objectsin an image (or slice) forms a region. A number of neighboring highdensity objects in another image (or slice) forms another region.

FIG. 2 shows an exemplary dental CBCT volume that contains three imageslices S1, S2, and S3. High density object examples are objects O1 andO2 shown in slice S1; these are parts of two adjacent or neighboringteeth. High density objects including objects O1 and O2 in S1 constitutea region in S1. Similarly, high density objects like O1 and O2 in S2constitute a region in S2. The same applies to S3.

There is a gap G1 between objects O1 and O2 in S1. The method of thepresent invention provides ways to identify a separation curve thatpasses through gap G1, optionally following initial user input andconditions, including identification of appropriate regions, asdescribed subsequently.

In slice S1, the high density objects (teeth in this case) arecollectively arranged in a geometric arch shape that can be decomposedinto a set of concentric curves. FIG. 3A shows an exemplary set ofconcentric curves including curves C1 and C2 in slice S1. Therequirement for forming a set of concentric curves is that these curvesshould cover (enclose) the region that is formed from the high densityobjects. An exemplary region R1 is shown in S1, encompassing the teethin this slice. Similarly, a corresponding arch-shaped region, withcorresponding curves, is formed for the region that contains teeth inimage slices S2 or S3.

Therefore, in a curve-forming step 106 of the FIG. 1 sequence,concentric curves are formed over the at least one object of interest.These curves are used for generating teeth separation curves forseparating teeth in the subsequent steps. Using these concentric curvesin slice S1 and in corresponding regions in slices S2, S3, and otherslices, a curved slab can be formed as a stack of these regions. Thecurved slab can then be cropped from the image volume for furtherprocessing.

As shown schematically in FIG. 3B, by stacking the regions that aredefined along these concentric curves, that is, stacking of region R1from slice S1 in FIG. 3A and corresponding regions R2, . . . Rk fromslices S2, . . . Sk that would be defined in the same way, a curved slabcan be formed as a curved sub-volume 130, containing one or more of thefeatures of interest; here, the features of interest are regions of oneor more high density objects cropped from the larger image volume.

The diagram of FIG. 3B shows schematically how the segmentation sequenceof the present invention proceeds to generate one or more panoramicviews 140 from a dental CT volume 120. A first set of operations,through step 106 in FIG. 1, generates the curved slab of curvedsub-volume 130, from the original CT volume 120. An unfold linecomputation step 108 then provides a utility that will be usedsubsequently for unfolding the curved sub-volume 130 along a selectedcurve to generate the desired flattened or unfolded panoramic view 140.In its effect, this maps the leaf of the foliation back to a plane,which can be readily manipulated and viewed as an image. As the sequenceshown in FIG. 3B indicates, the curved sub-volume 130 is formed bystacking slices aligned generally along a first direction. Unfoldingthen operates in a planar direction that is orthogonal to this firstdirection, as shown in the view of an unfolded slab, termed an unfoldedsub-volume 134. For unfolding, image data elements that lie along ornearby each fold line are re-aligned according to a realignment of thefold lines. This realignment generally aligns the fold lines from theirgenerally radial arrangement to a substantially parallel orientation.Image data elements that were initially aligned with the fold lines inthe original, generally radial arrangement follow the fold linere-orientation, effectively “flattening” the curved sub-volume withlittle or no distortion of the tooth and its position relative to otherteeth.

Unfolded sub-volume 134 can be visualized as a stacked series ofvertical slice images V1, V2, . . . Vj, as shown in FIG. 3B. Eachvertical slice provides a panoramic image obtained at some depth withinunfolded sub-volume 134. Subsequent steps then present the unfoldedviews to the user as a type of index to the volume that is to beassigned separation curves. That is, selection from the unfolded viewenables the user to provide hint (or seed) information that is used fordissection or separation and, in most cases, subsequent segmentation ofthe tooth or other object.

The one or more concentric curves or curved paths in FIG. 3A could betraced using an automated approach or a semi-automatic approach. In anautomated approach, slice S1 can be processed through a sequence ofsteps that include noise filtering, smoothing, intensity thresholding,binary morphological filtering, medial curve estimation, and pruning toidentify a first curve that fits or approximates the arch shape of theteeth region. Subsequent concentric curves can then be defined using theshape and position of the first estimated curve as a starting point.These steps described are exemplary steps that are well known to thoseskilled in the art; other manual and automated processing steps couldalternately be performed for providing a structure to support unfolding.

A semi-automatic approach can be simpler and more robust, withoutrequiring an elaborate operator interface. For such an approach, userinput initializes a few nodes along an imaginary medial axis of the archshape region in slice S1 (FIG. 3A, 3B), for example. These few nodesthen become starting points for a curve-fitting algorithm, such as aspline-fitting sequence, for example, to form a first curve that fitsthe arch shape of the teeth region. Subsequent concentric curves canthen be generated using the first estimated curve. Steps foroperator-assisted curve definition and generation of parallel orotherwise related curves are familiar to those skilled in the imageanalysis arts.

Once the concentric curves are formed in step 106 (FIG. 1), computationstep 108 computes lines that are generally perpendicular to theconcentric curves in the tomographic image space. These perpendicularlines facilitate finding separation curves between the teeth insubsequent processing. Exemplary perpendicular lines are shown as unfoldlines L1 and L2 in FIG. 4. It is readily understood that two neighboringperpendicular lines could touch or intersect at one end but be spacedapart at the other end by examining the exemplary perpendicular lines inslice S1 in FIG. 4.

In an unfolding step 110 (FIG. 1) the curved slab containing one or moreof the regions of one or more high density objects is unfolded with thehelp of the computed unfold lines that are perpendicular to theconcentric curves.

The logic flow diagram of FIG. 5A shows the sequence for unfolding thecurved slab according to an embodiment of the present invention. In adefinition step 250, an x-y coordinate system for slice S1, as shown inFIG. 4, is defined. The origin is at the upper left corner of slice S1.Suppose there are a total of M concentric curves (C1, C2, . . . Cm, . .. CM), and a total of N perpendicular lines (L1, L2, . . . Ln, . . .LN). An x position matrix of size of M by N is denoted by X. A yposition matrix of size of M by N is denoted by Y. A storage step 254stores the x position of an intersection point of Cm and Ln at matrixX(m,n). The y position of an intersection point of Cm and Ln is storedat matrix Y(m,n). In a slice assignment step 260, an arbitrary slice Sis denoted with the same x-y coordinate system defined in definitionstep 250 and shown in FIG. 4.

Continuing with the FIG. 5A sequence, in an intensity image step 264 anarbitrary intensity image by U of size of M by N is generated. Define:U(m,n)=S(Y(m,n),X(m,n)).

Therefore, for a specific slice, a series of intensity images aregenerated:U1(m,n)=S1(Y(m,n),X(m,n))U2(m,n)=S2(Y(m,n),X(m,n))U3(m,n)=S3(Y(m,n),X(m,n)), etc.

Collectively, the intensity images U1, U2, U3, etc. formed in this wayconstitute an unfolded curved slab. Then, in an average image step 270(FIG. 5A), having the unfolded curved slab ready, computing an averagedimage W of the unfolded curved slab in the axial direction yields:W=(U1+U2+U3+ . . . +UK)/K,where K is the number of slices contained in the unfolded curved slab.

FIG. 5B is a logic flow diagram that shows a sequence of steps forgenerating and displaying a separation curve between a first and asecond object, such as between two teeth, in a volume image. FIG. 5Cshows an imaging apparatus 330 that is used for the steps shown in FIG.5B. Imaging apparatus 330 has an image acquisition apparatus 332 such asa CBCT imaging system, for example, that provides the volume image ofthe patient. The volume image data may be obtained directly from imageacquisition apparatus 332 or from a memory 336 or other storage system,such as a PACS (picture archiving and communication system) or othersystem that stores the acquired image data. Imaging apparatus 330 has acomputer system or other logic processor 334 with a display console 338that provides both display and operator interface functions, such asthrough a keyboard 320 and mouse 322 or incorporating a touch screen,for example. In an image acquisition step 340, the volume image data ofthe patient or other subject is acquired from memory 336 or othersource.

Continuing with the FIG. 5B sequence, a region identification step 344then identifies the region of the volume image that contains at leastthe first and second teeth or other objects. A curve tracing step 348then forms one or more concentric curves over at least the first andsecond objects in the region according to the volume image data. Acomputation step 350 then computes one or more lines perpendicular tothe one or more concentric curves. An unfolding step 354 unfolds theregion covered by the one or more concentric curves to generate anunfolded view of the region. An obtain outline step 358 obtains anoutline of a geometric primitive that is traced with respect to thefirst object in the unfolded region, based on operator input. Entry of ageometric primitive by the operator is described in more detailsubsequently. A profile line generation step 360 then generates aplurality of profile lines across the first and second objects usinginformation obtained from the outline of the geometric primitive that isobtained in step 358. Gap points in each profile line are thenidentified and joined together in a separation curve forming step 364.Separation curves are then displayed relative to the volume image in adisplay step 370.

FIGS. 6A through 6D show various features of the process and show howthe geometric primitive that is entered by the operator is used toprovide starting points for tooth dissection. FIG. 6A shows views V0-VMfor different unfolded views of the teeth. View V0 is the averaged viewthat is obtained using the data in each slice V1-VM. Slices V1, V2, andVM are individual coronal view slices, generated as was describedearlier with reference to FIG. 3B. Although a geometric primitive can betraced onto any one or more of the individual slices of views V1-VM, itis generally preferable to have the operator trace a geometric primitiveonto the averaged view V0. Although the operator can trace the geometricprimitive onto any view of the unfolded volume image, the coronal viewis advantaged for ease of visualization and usability.

FIG. 6B shows operator entry of a box 532 as one type of geometricprimitive 534 that helps to identify starting points for toothdissection. Box 532 can be readily entered with a computer mouse in oneaction, using standard drag-and-hold procedure to define diagonalcorners, for example. The geometric primitive that is entered defines aheight 186 and edges for providing start points for dissectionprocessing. The height that is defined helps to limit the amount of datathat is processed in subsequent steps, so that volume image data outsideof the region of the teeth or other objects can be ignored for thispurpose. Alternate types of geometric primitive include points, lines,circles, or free-form closed or open figures or shapes, for example.

The schematic diagram of FIG. 6C shows, from a perspective view ofvertical (coronal) slices V1-Vj, how edges of the geometric primitive534 are used. Extreme edges or sides of geometric primitive 534 areextended to define extended lines 190 that can be used to identifystarting points f0 a and f0 b for each vertical slice V1-Vj. Startingpoint f0 a is shown on the first slice V1; starting point f0 b is shownin the middle of the stack of coronal slices. In practice, the startingpoints can be identified at any point along the extended line 190, thatis, in reference to the designations shown in FIG. 6C, in the image datacorresponding to any of vertical slices V1-Vj.

FIG. 6D shows an axial view averaged image W corresponding to thecoronal view schematic of FIG. 6C. Image W has size M by N pixels. Inunfolded FIG. 6D, the high density objects (teeth) are alignedapproximately along a horizontal direction instead of along a curvedmedial axis as was shown in FIG. 2. Lines drawn in the horizontaldirection correspond to the top edges of slices V1-Vj in FIG. 6C. Threerepresentative profile lines p1, p2, and p3 are indicated for a numberof the slices in this view.

The unfolding operation of the curved slab, as described earlier withreference to FIG. 3B, makes it possible to trace separation curvesbetween the teeth in a more straightforward manner, as described in moredetail later. Objects O1 and O2 in FIG. 2 correspond to objects Q1 andQ2 in FIG. 6D; the shape of either O1 or O2 may be transformed to Q1 andQ2 due to the unfolding operation. A gap G1 between O1 and O2 in FIG. 2now appears as gap H1 in the unfolded view of FIG. 6D. Given thesetransformations, the task of automatically finding a separation curvethat passes through gap G1, with or without an initial condition imposedby the user, then becomes the task of finding a separation curve thatpasses through gap H1. This task is then more easily accomplished bysearching along the same direction indicated by an arrow 520 for eachpair of teeth in image W in FIG. 6D.

With reference to the sequence of FIG. 1, identifying the separationcurve between two teeth in image W is carried out in a curveidentification step 112 by identifying points of minimum intensity, orvalley points along profile lines. As noted with respect to FIGS. 6C and6D, profile lines correspond to the top edges of each vertical slice(viewed along the top in the perspective view of FIG. 6C and from thetop in the plan view of FIG. 6D.) A point of minimum intensitycorresponds to an image element, voxel or pixel, at a position in theimage data. The use of profile lines is a convenience and helps tovisualize the direction in which processing progresses in order toidentify the succession of valley points needed to define the separationcurve.

FIGS. 7A and 7B show, in schematic form, how the separation curve d1between objects Q1 and Q2 is formed by connecting points in anincremental fashion. FIG. 7A is a plan view, showing multiple profilelines p1, p2, p3, and so on. FIG. 7B shows, from a perspective view, howprofile lines within a section identified as inset E in FIG. 7A, providevalley points f1, f2, f3, and so on. An initial starting point f0 isidentified from an edge of a geometric primitive traced by the operator,as was described earlier with reference to FIGS. 6B-6D. In the exampleshown, analysis of the image data along profile line p1 indicates that anearby point f1 is a more suitable valley point and is thus substitutedfor f0 as the effective starting point for forming separation curve d1.Then, progressing from point f1 in the direction of arrow 520 (the ydirection) and searching along the next profile line p2, the next valleypoint f2 is identified. According to an embodiment of the presentinvention, a constraint is imposed on how far the next valley point canbe displaced in the +/−x-direction (orthogonal to arrow 520, as shown inFIG. 7A) with each step in the y direction. This constraint is in the xdirection that is substantially orthogonal to the path of the separationcurve at any valley point. According to an exemplary embodiment of thepresent invention, in moving in the y direction from one profile line tothe next, the x-coordinate of the next valley point must be within +/−6voxels (or pixels) of the x-coordinate for the preceding valley pointthat has been identified. It can be appreciated that this constraintvalue helps to prevent abrupt changes in the overall direction of theseparation curve and can be adjusted appropriately for controlling thepath of the separation curve.

The same process repeats until all the profile lines pn are searched.The collection of valley points including f1, f2 and f3 are connected toform a separation curve d1 that separates teeth Q1 and Q2 or otheradjacent objects. In the same manner, a separation curve on the otherside of a tooth is similarly generated. The process repeats as often asneeded until all needed pairs of teeth are equipped with separationcurves.

Then, referring back to FIG. 1, a mapping step 114 maps the separationcurves back to the computed tomographic image space as shown in FIGS. 8and 9. FIG. 8 is the unfolded view of averaged image W. FIG. 9 is a viewof a reconstructed CBCT volume image having separation curves K1 and K2mapped between the objects of interest. Recalling the definitions ofX(m,n) and Y(m,n) that are stated previously, a vector (m,n) can bereadily mapped to (Y(m,n), X(m,n)) in the computed tomographic imagespace. A pair of separation curves d1 and d2 in FIG. 8 correspond tocurves K1 and K2 in FIG. 9. These mapped separation curves can then bedisplayed on a control monitor associated with the computer logicprocessor or other processor that executes the procedures describedherein. Separation curves can be displayed in a 2D or 3D rendering ofthe associated image data. In a volume rendering, a separation curve canbe viewed from any suitable angle.

Now referring to FIG. 10, there is shown a logic flow diagram thatdescribes an auto-segmentation sequence according to an embodiment ofthe present invention. A digital dental volume such as a CBCT volume isacquired in a data acquisition step 1002. The upper teeth (jaw) andlower teeth (jaw) are the regions of particular interest. In general,teeth have higher code values than most of the other tissues and bonesin a CBCT volume. An adaptive thresholding approach, for example, canquickly isolate the upper and lower regions from the rest of the imagecontent. Adaptive thresholding is an approach for analyzing andsegregating image content that is familiar to those skilled in imageprocessing. With this initial processing, the next step is to separateupper and lower jaws in a jaw separation step 1004.

The boundary of separation between the upper and lower jaws is readilydetected by evaluating the code value profile in the sagittal or coronalviews of the jaw regions, or in a panoramic view, such as the unfoldedview shown in FIG. 6B and generated as described earlier with referenceto FIG. 3B. For instance, in FIG. 6B, the lower code values in thatportion of an image between the upper teeth and lower teeth provide agood indication of separation between the upper jaw and the lower jaw.Accumulated information from lower code values for successive imageslices or, alternately, for a portion of the volume image content,facilitates the task of identifying and differentiating upper and lowerjaws in the volume image.

After separating the upper and lower jaws, the sequence of FIG. 10 nextestimates the average teeth height for the upper teeth and lower teethrespectively in an estimation step 1006. A simple and quick approachemployed in the present invention is, for each tooth in the upper jaw orlower jaw, to project all the teeth voxels to a single line, thenevaluate the projection profile. For instance, referring to the unfoldedview of FIG. 11A, generated as described earlier with reference to FIG.3B, projecting the lower teeth voxels either from left to right or fromright to left in an image 1104 generates a line profile 1108 which, inturn, shows the length of the teeth projected onto the line. A line 1110indicates a low point in line profile 1108 that indicates the gapbetween lower and upper teeth. A second line 1112 indicates a relativelow line marking the base of the teeth. The distance between lines 1110and 1112 yields a length that indicates the average height for upperteeth or lower teeth. An optional line 1114 indicates the approximatelocation of the gumline; the distance between lines 1114 and 1110indicates approximate height of the crown. The average height is usedsubsequently to define a bounding cube for a tooth volume.

A jaw arch detection step 1008 in the FIG. 10 sequence performs similartasks to those performed for unfolding a region in steps 106 through 110(FIG. 1), and then proceeds to detect the jaw arch region for a volume,such as for a volume image 1102 represented in FIG. 11B. Results of step1008 are shown in the example of an image 1202 in FIG. 12, where lines1204 indicate the detected jaw arch region.

A line identification and mapping step 1010 in FIG. 10 performs similartasks to that of steps 112 and 114 in FIG. 1, finding the dissection orseparation curve between the teeth. It should be noted that theseparation curve is often generally linear, but may have some number ofangled or curvilinear segments. Unlike the FIG. 1 procedure, however, instep 1010, the initial starting point f0 is not identified from an edgeof a geometric primitive traced by the operator; instead, the initialstart point f0 is detected automatically by the software. A searchroutine searches for the valley points (locally lowest code values) onthe medial line of the jaw arch region. These valley points can be usedas the initial start point f0s.

The purpose of finding the separation curves is to adequately define abounding cube as a sub-volume for an individual tooth within the volumeimage. In order for a bounding cube to completely enclose a tooth, theseparation curve that is initially identified is shifted a few voxels inposition away from a bounded region. For example, in FIG. 13, an image1302 is shown. For Tooth A, one of the original dissection or separationcurves is a curvilinear line 1304, the other separation curve(separating Tooth A and Tooth B) is a curvilinear line 1306. For thebounding cube boundary for Tooth B, the separation curve that is used(separating Tooth B and Tooth A) is a curvilinear line 1308, shiftedoutward from the original curvilinear line 1306.

For the example image shown in FIG. 13, the axial view of bounding cubesis shown in an image 1402 in FIG. 14. With reference to both FIGS. 13and 14, a rectangle 1404 for Tooth A is determined by separation curves1304 and 1306. A rectangle 1406 for Tooth B is determined by separationcurves 1308 and 1310. Again, rectangle 1404 is an axial view of thebounding cube that provides a sub-volume for segmentation of Tooth A,the height of the bounding cube for Tooth A is the average teeth heightestimated in step 1006 (FIG. 10). Similarly, rectangle 1406 is an axialview of the bounding cube that provides the sub-volume for segmentationof Tooth B, the height of the bounding cube for Tooth B is the averageteeth height estimated in step 1006. In a tooth volume definition step1012 in FIG. 10, these bounding cubes define individual teeth volumes,or sub volumes, to be used in subsequent segmentation processing.

Following the steps for defining a tooth volume (or sub volume) thesegmentation process of a tooth object of interest is carried out in theFIG. 10 sequence in a seed generation step 1014 and a segmentation step1016. For segmentation, a sub volume SV is presented to an imageprocessing algorithm; this may be a bounding cube as previouslydescribed. The image processing algorithm in step 1014 then provideslabeling hints, such as seeds, to a segmentation algorithm. Thealgorithms in step 1014 can be a combination of filtering, edgedetection, pattern recognition, thresholding, morphological processing,classification, line detection, ellipse detection, image masking, orother steps for generating foreground and background seeds that provideinitial definition of foreground (tooth) from background data. With thisdata provided as foreground and background seeds, for example, regiongrowing logic can be applied to the overall image as an initial part ofthe segmentation process. Region growing using seed data as labelinghints is an image segmentation method that is well known to thoseskilled in the image segmentation arts.

The exemplary labeling hints that are generated for seed generation step1014 can contain location or position hints for objects of interest(tooth voxels in this context) instead of or in addition to imageintensity hints for objects of interest. In addition, these hints canalso include location hints for background content, or objects ofnoninterest, image intensity hints for objects of noninterest, and otherrelated content. Segmentation methods for teeth and other structuresusing various types of labeling hints are well known to those skilled inthe dental image analysis arts. A segmentation algorithm such as themethod disclosed in commonly assigned US Patent Application PublicationNo. US2012/0313941 by Li et al, entitled “System and method for highspeed digital volume processing”, incorporated herein by reference inits entirety, can be used. Segmentation can be applied in step 1016 tosegment one or more teeth, or parts of a tooth or teeth. FIG. 15 showsan exemplary tooth segmentation result of an embodiment of the presentinvention in an image 1502.

In general, the results of step 1016 are presented to the user in agraphical display, either two dimensionally or three dimensionally.These results may be presented as optional, for approval by theoperator, for example. The user has an option to modify the results byadding or removing labeling hints (such as by editing parameters) andresubmitting the hints to step 1016 for another round of segmentation.Threshold values for pixel or voxel intensity, for example, can beedited for either foreground (tooth) or background content. This processcan be repeated a plurality of times until the user is satisfied withthe segmentation results.

Now referring to FIG. 16 there is shown a flowchart that describes thesteps of teeth alignment detection of the present invention. A digitaldental volume such as a CBCT volume is acquired in a data acquisitionstep 1602. A teeth segmentation step 1604 performs tasks similar topreviously described steps 1004 through 1016 of FIG. 10 to obtain a setof segmented teeth for either upper jaw or lower jaw as shown in FIG.15.

In a centroid computation step 1606, the positions of centroids ofsegmented teeth are computed. FIG. 17 shows an image 1702 with exemplarycentroids 1704. Here, a centroid 1704 of a tooth is the intersectionpoint of a tooth crown surface and a tooth principal axis 1708, as shownin FIG. 17. A principal axis is one of the eigenvectors computed usingthe 3D positions of all or part of the voxels of a tooth and the codevalues of all or part of the voxels of a tooth. Alternately, thecentroid of a tooth can also be computed as the center of inertia,alternately termed the inertia center, of a segmented tooth, or theinertia center of the crown part of a tooth, that portion above the gumline. The inertia center computation takes into account differences inmass for tooth content as well as surface shape characteristics. Acentroid of a tooth can alternately be computed as the spatial center or3D geometric center of a segmented tooth, or as the geometric center orapproximate geometric center of the crown portion of a tooth. Thus, itcan be seen that there are a number of alternative ways to calculate thecentroid for a segmented tooth. According to an embodiment of thepresent invention, the same centroid computation is used for each of thesegmented teeth, whether the centroid is considered as the intersectionof the tooth principal axis with the surface of the crown, the inertiacenter of the tooth or tooth crown portion, or the spatial or 3Dgeometric center of the tooth or tooth crown portion. That is, providedthat the centroid for each tooth in a set of teeth is computed in thesame way, the pattern of centroids for the set of teeth can be analyzedand used, as described subsequently. Methods for eigenvector, inertiacenter, and tooth centroid computation are well known to those skilledin the orthodontic measurement, cephalometric, and evaluation arts andare applied, for example, in using a number of methods used fororthodontics and cephalometric assessment.

The centroids computed in step 1606 are subject to alignment assessmentin an alignment detection step 1608. In general, centroids of a set ofteeth, either upper teeth or lower teeth, provide spatial referencepoints that define extreme points of a convex hull, as the term isunderstood to those skilled in the imaging analysis arts. Extreme pointsare points that lie along the boundary of the convex hull. In practice,the convex hull defines a smooth unbroken curve that changes directionpredictably, and does not have sudden indented or out-dented portions.In a convex hull construction, a straight line connecting any two pointswithin the convex hull also lies fully within the convex hull. FIG. 18shows a convex hull 1802 defined by a number of centroids 1704.According to an embodiment of the present invention, if all of thecentroids of the set of three or more teeth teeth (either upper or lowerjaw) lie along a convex hull as shown in FIG. 18, then the set of teethare considered to be aligned. Convex hull conformance is thus used tohelp identify tooth misalignment.

FIG. 19 shows an image 1902 with an exemplary mis-aligned set of teeth.FIG. 20 shows only the centroids 1704 and 1904 of these teeth. Tooth C,with its centroid 1904, is noticeably shifted from its originalposition, such as is shown in FIG. 17. Applying convex hull detectionalgorithm to this altered set of centroids reveals that centroid 1904 isno longer an extreme point of a convex hull 2002 as shown in FIG. 20.

According to an embodiment of the present invention, if one or more ofthe centroids are not extreme points of a convex hull, mis-alignment isdetected for a particular set of teeth. In the workflow of FIG. 16, theassessment results are reported to the user in a reporting step 1610.This condition is reported by a display of teeth and centroids, forexample, with the centroids that fail to meet the convex hull criterionhighlighted on the display screen. The user can then digitallymanipulate the one or more mis-aligned teeth through a graphical userinterface (not shown) in order to shift the position of the misalignedtooth towards or away from alignment (step 1612). The computer algorithmthen repeats step 1608 in real time and reports the results. For FIG.19, for example, an operator interface instruction, entered using amouse, a keyboard arrow key, or a touch screen, incrementally shifts theposition of the misaligned tooth toward a more suitable, alignedposition. Alternately, system software automatically shifts the positionof a centroid that is indicative of misalignment, moving the centroidand its corresponding tooth on the display until an improved alignmentis achieved.

Embodiments of the present invention provide methods for assisting thedental practitioner in assessing tooth misalignment using volume imagedata. By computing and using centroids of segmented teeth, these methodsprovide a straightforward way to present alignment information to theviewer.

According to an embodiment of the present invention, a computer programhas stored instructions that process image data accessed from anelectronic memory in accordance with the method described. As can beappreciated by those skilled in the image processing arts, a computerprogram of an embodiment of the present invention can be utilized by asuitable, general-purpose computer system, such as a personal computeror workstation. However, many other types of computer systems can beused to execute the computer program of the present invention, includingnetworked processors. The computer program for performing the method ofthe present invention may be stored in a computer readable storagemedium. This medium may comprise, for example; magnetic storage mediasuch as a magnetic disk (such as a hard drive) or magnetic tape; opticalstorage media such as an optical disc, optical tape, or machine readablebar code; solid state electronic storage devices such as random accessmemory (RAM), or read only memory (ROM); or any other physical device ormedium employed to store a computer program. The computer program forperforming the method of the present invention may also be stored oncomputer readable storage medium that is connected to the imageprocessor by way of the internet or other communication medium. Thoseskilled in the art will readily recognize that the equivalent of such acomputer program product may also be constructed in hardware.

It will be understood that the computer program product of the presentinvention may make use of various image manipulation algorithms andprocesses that are well known. It will be further understood that thecomputer program product embodiment of the present invention may embodyalgorithms and processes not specifically shown or described herein thatare useful for implementation. Such algorithms and processes may includeconventional utilities that are within the ordinary skill of the imageprocessing arts. Additional aspects of such algorithms and systems, andhardware and/or software for producing and otherwise processing theimages or co-operating with the computer program product of the presentinvention, are not specifically shown or described herein and may beselected from such algorithms, systems, hardware, components andelements known in the art.

The invention has been described in detail with particular reference topresently preferred embodiments, but it will be understood thatvariations and modifications can be effected that are within the scopeof the invention. For example, geometric shapes entered by the operatormay have a default shape, such as a rectangle of a predefined size.Operator instructions or overrides can be entered in any of a number ofways. Volume image data can be obtained from CBCT and visible lightimaging. The presently disclosed embodiments are therefore considered inall respects to be illustrative and not restrictive. The scope of theinvention is indicated by the appended claims, and all changes that comewithin the meaning and range of equivalents thereof are intended to beembraced therein.

The invention claimed is:
 1. A method of reporting alignment of teeth,executed at least in part on a computer system and comprising: acquiringvolume image data for at least one of the upper and lower jaw regions ofa patient; segmenting three or more teeth from the acquired volume imagedata; for each segmented tooth, identifying a centroid position;generating and displaying a convex hull wherein the convex hull hasextreme points corresponding to centroid positions for the three or moresegmented teeth; and highlighting one or more of the centroids that failto conform to the generated convex hull.
 2. The method of claim 1,wherein identifying the centroid position comprises identifying aninertia center of at least a portion of the segmented tooth.
 3. Themethod of claim 1, wherein identifying the centroid position comprisesidentifying a geometric center of at least a portion of the segmentedtooth.
 4. The method of claim 1, wherein highlighting the one or morecentroids that fail to conform to the generated convex hull comprisesdisplaying the one or more centroids in a color.
 5. The method of claim1, further comprising shifting the position of the one or morehighlighted centroids toward a position that conforms more closely tothe generated convex hull.
 6. The method of claim 1, further comprisingaccepting an operator instruction for shifting the position of the oneor more highlighted centroids toward a position that conforms moreclosely to the generated convex hull.